Field of the Subject Disclosure
The present subject disclosure relates to image analysis. More particularly, the present subject disclosure relates to automatically identifying structures (e.g., cellular structures) or patterns (e.g., background or white space) in an image.
Background of the Subject Disclosure
In the analysis of biological specimens such as tissue sections, blood, cell cultures and the like, biological specimens are often stained with one or more combinations of stains or assays, and then the stained biological specimen is viewed or imaged for further analysis. Observing the assay enables a variety of processes, including diagnosis of disease, assessment of response to treatment, and development of new drugs to fight disease.
For example, upon applying a light source to the tissue, the assay can be assessed by an observer, typically through a microscope. Alternatively, an image may be generated of the biological specimen after and assay has been applied, and image data can be acquired from the assay for further processing. In such an acquisition, multiple channels of image data, for example RGB color channels, are derived, with each observed channel comprising a mixture of multiple signals. Processing of this image data can include methods of color separation, spectral unmixing, color deconvolution, etc. that are used to determine a concentration of specific stains from the observed channel or channels of image data. For image data processed by automated methods, depicted on a display, or for an assay viewed by an observer, a relation may be determined between a color of the tissue and a color of the stains, to determine a model of the biomarker distribution in the stained tissue. A local presence and amount of stain may indicate a presence and a concentration of the biomarkers queried in the tissue.
The publication ‘Adaptive Spectral Unmixing for Histopathology Fluorescent Images’ by Ting Chen et al, Ventana Medical Systems, Inc. provides an introduction and an overview as to various prior art techniques for spectral unmixing of multiplex slides of biological tissue samples, the entirety of which is herein incorporated by reference. Various other techniques for spectral unmixing of tissue images are known from WO 2012/152693 A1 and WO 2014/140219 A1.
Multiplex immunohistochemistry (IHC) staining is a technique for the detection of multiple biomarkers within a single tissue section and has become more popular due to its significant efficiencies and the rich diagnostic information it generates. IHC slide staining can be utilized to identify proteins in cells of a tissue section and hence is widely used in the study of different types of cells, such as cancerous cells and immune cells in biological tissue. For example IHC staining may be utilized in the diagnosis of abnormal cells such as the ones in cancerous tumors. Typically, the immunological data indicates the type, density, and location of the immune cells within tumor samples and this data is of particular interest to pathologists in determining a patient survival prediction. Thus, IHC staining may be used in research to understand the distribution and localization of the differentially expressed biomarkers of immune cells (such as T-cells or B-cells) in a cancerous tissue for an immune response study. For example, tumors often contain infiltrates of immune cells, which may prevent the development of tumors or favor the outgrowth of tumors. In this scenario, multiple stains are used to target different types of immune cells, and the population distribution of each type of immune cell is used in studying the clinical outcome of the patients.
Immune profile studies typically relate the immune response to the growth and recurrences of human tumors. However, a prerequisite of the immune profile study requires the human observer, utilizing a brightfield microscope, to manually locate and count the number of different immune cells within the selected tissue regions, for example, the lymph node regions which may contains hundreds to thousands of cells. This is an extremely tedious and time consuming process and the results may also subject to intra- and inter-individual variability. A tissue slide is typically stained by the IHC diagnostic assay with the cluster of differentiation (CD) protein markers identifying the immune cells and the nucleus marker Hematoxylin (HTX) marking the nuclei. The stained slide is then imaged using a CCD color camera mounted on a microscope or a scanner. The acquired RGB color image is hence a mixture of the immune cell membrane and the universal cell nuclear biomarker expressions.
Several techniques have been disclosed in the prior art to detect the cells. Most of the techniques are based on image processing that capture the symmetric information of the cell appearance features. Machine learning techniques have also been explored for cell detection, such as statistical model matching learned from structured support vector machine (SVM) to identify the cell-like regions. However, these techniques are limited to automatic nucleus detection rather than membrane detection. Since immune cell markers such as CD3 and CD8 for universal T-cells and cytotoxic T-cells respectively are membrane markers, the stain shows a ring appearance rather than the blob appearance of a nucleus. Although some machine learning based systems use scale invariant feature transform (SIFT) for maintaining sufficient contrast of cell boundaries, this method was developed for unstained cell images and it is non-trivial to extend it to detect immune cells in IHC stained images.